Boosting for multiclass semi-supervised learning

Supervised learning methods are effective when there are sufficient labeled instances. In many applications, such as object detection, document and web-page categorization, labeled instances however are difficult, expensive, or time consuming to obtain because they require empirical research or experienced human annotators. Semi-supervised learning algorithms use not only the labeled data but also the unlabeled data to build a classifier. The goal of semi-supervised learning is to use unlabeled instances and combine the information in the unlabeled examples with the explicit classification information of labeled examples for improving the classification performance. Most of the semi-supervised learning algorithms were designed for binary classification problems. However, many practical domains, for example recognition of speech, objects, and characters, involve more than two classes. A Multiclass classification problem can be decomposed into a number of independent binary classification problems by utilizing methods like one-versus-all. However, these schemes have their problems. One-versus-all results in imbalanced distributions. Since each classifier is trained independently, the weights of their outputs may be on different scales, so that combining them is non-trivial. There is thus a need for direct multiclass algorithm for semi-supervised learning. In this paper we propose a new algorithm for Multiclass semi-supervised learning that follows the boosting approach and is a direct generalization of the binary SemiBoost algorithm [3], which uses both the similarity between the points and the classifier predictions to sample and assign “pseudo-labels” to the unlabeled examples, to the multiclass setting, named as Multiclass SemiBoost. The key advantage of Multiclass SemiBoost is to exploit both the manifold and the cluster assumption to train the classifiers using boosting. We derive the algorithm from an objective function that combines empirical loss on the labeled data and inconsistency of